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and Assessment System for the Tri-County .Charter School Partnership.” This 
partnership of five charter schools was working to develop a replicable and 
comprehensive accountability system for each of the schools. A mixed method 
research design was developed using parallel or simultaneous methodologies. 
Quantitative methodology provided the primary data for assessing outcomes 
related to student achievement, and qualitative methodology provided 
additional data on outcomes while adding depth and breadth to the evaluation. 
Surveys completed by 793 students, 335 parents, and 21 teachers were used to 
obtain, information, workshop evaluations gave insight into what participants 
found useful and effective, and interviews with 5 administrators provided 
depth and insight into the context in which the program was conducted. Taken 
together, findings indicate that the partnership was successful in 
establishing the groundwork for year-two activities. The schools collaborated 
in providing professional development and aligned their curricula with 
Florida state standards. No computer-based student progress and standards 
reporting system had yet been developed, but it was recognized that a student 
database was essential for tracking student performance on standardized tests 
and performance assessments. Stakeholder surveys showed very positive 
attitudes and provided information for the second year of program activities. 
Although no comprehensive accountability system has yet been developed for 
the charter school system, the project has provided the framework for the 
schools to develop such a system. (Contains 17 references.) (SLD) 
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Background 

Concern that no appropriate means existed to measure how successful charter schools 
were at improving students’ performance, a partnership of five charter schools in Dade, 
Broward, and Palm Beach counties (Tri-County Charter School Partnership) applied for and 
received a South Florida Annenberg Challenge Implementation Grant. Given the high 
expectations for student performance driven by state standardized tests (FCAT), the member 
schools proposed the development and implementation of a replicable, comprehensive 
accoimtability system in each of the partnership charter schools. This paper shares the results 
of the first-year evaluation of a two-year project, “Comprehensive Accountability, 
Professional Development, and Assessment System for the Tri-County Charter School 
Partnership.” 



Program Goal 

The goal for this project is to improve student achievement through the development 
and implementation of a replicable, comprehensive accoimtability system in each of the 
partnership schools. This goal would be accomplished through the autonomous leadership at 
each school under the guidance and direction of the principal. This decentralized decision 
making allows each school to have ownership of the program (Schwahn & Spady, 1998). 




Program Objectives 

To support the partnership in achieving its goal, six project components were 
identified and restated to serve as program objectives. Each of these objectives supports the 
formation of the accountability system. The purposes of this project are to: 

1 . Provide the opportunity for professional development activities to prepare teachers 
and a dmini strators with the skills necessary to develop standards-based accountability 

, systems with performance assessments aligned to these standards. 

2. Align the curriculum at each charter school with the Florida Sunshine Standards and 
other agreed upon standards and benchmarks. 

3. Create and utilize appropriate classroom performance assessments. 

4. Design and implement a standards based student progress report. 

5. Implement a computer-based system to provide feedback to students, teachers, 
administrators, and parents on student progress in meeting these standards. 

6. Implement a parent, student, and community satisfaction survey. 



Evaluative Questions 

To focus the direction of this evaluation, six evaluative questions were developed. 

\ These evaluative questions were derived from the program’s objectives. The questions and 
^ the sub-questions generated from them broadly assess the foci of the project. 

Evaluative Question 1. Did the program provide opportunities for professional 
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development activities that prepare teachers and administrators with the skills necessary 
to develop a standards-based accountability system and performance assessments aligned 
to these standards. 

Sub-question 1.1. Did teachers and administrators attend the professional 
development workshops? 

Sub-question 1.2. Did participants perceive that the professional development 
workshops provided them with the information they needed to meet program 
objectives? 

Sub-question 1.3. What other forms of professional development activities did the 
participants feel would be helpful in meeting program objectives? 

Evaluative Question 2. Did each charter school align their curriculum with the Florida 
Sunshine Standards and other agreed upon standards and benchmarks? 

Sub-question 2.1. Were power standards identified? 

Evaluative Question 3. Did each charter school create and utilize appropriate classroom 
performance assessments? 

Sub-question 3.1. Were customized performance assessments purchased jfrom the 
Center for Performance Assessment? 

Sub-question 3.2. Was the International Performance Assessment System (IP AS) 
purchased? 

Sub- question 3.3. Was the IP AS (performance assessments) utilized by each charter 
school? 

Sub-question 3.4. Did each charter school create their own customized classroom 
performance assessments? 

Sub-question 3.5. Did each charter school utilize their customized performance 
assessments? 

Evaluative Question 4. Was a Standards Based Progress Report created and 
implemented? 

Evaluative Question 5. Was a computer based student progress and standards reporting 
system purchased and utilized? 

Evaluative Question 6. Were stakeholder satisfaction surveys administered to students, 
parents, teachers, administrators, special program/resource teachers, and auxiliary 
personnel? 

Sub-question 6.1. What were the perceptions of the school of students, parents, 
teachers, administrators, special program/resource teachers, and auxiliary personnel? 
Sub-question 6.2. What are the stakeholder perceptions when the questionnaire 
results are broken down by sub-scale: administrative leadership, high expectations, 
school chmate, basic skills, monitoring pupil progress, and resources? 
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EVALUATION METHODOLOGY 



Evaluation Design 

In order to evaluate success in meeting the project goal and assess outcomes related to 
the objectives, a mixed method research design was developed using parallel or simultaneous 
methodologies (Tashakkori & Teddlie, 1998). This design was selected because it provided 
in-depth evaluation of the program, addressing concerns about validity and reliability and 
thus the transferability of the findings to other settings. In addition, the use of parallel 
quantitative and qualitative methodologies triangulates the data and strengthens the 
evaluative findings (Breen, Jenkins, Lindsay, & Smith, 1998). 

Quantitative methodology provided the primary data for assessing outcomes related 
to student achievement. This methodology used descriptive research to provide insight into 
the current status of the project and to gather baseline data for future research (Gay & 
Afrasian, 2000). Additionally, this design was selected because self-report instruments, 
observations, and record files were used for gathering data. 

Qualitative methodology provided additional data on outcomes while adding depth 
and breadth to the evaluation (Seale, 1 999). The qualitative methods of interviews, document 
analysis, and observation served as additional data sources to obtain participants’ 
perceptions. Documenting and analyzing these perceptions allowed the evaluation team to 
better understand the “how” and “why” behind project outcomes (Morgan & Krueger, 1993; 
Posner, 1995; Rubin & Rubin, 1995). 

Theoretical Framework 

In addition to an evaluation design, a theoretical framework was used to guide the 
evaluation of the project. It provided a structure for thinking about data and a basis for 
generalizing the qualitative findings (Seale, 1999). The theoretical firamework used for this 
evaluation was an adaptation of the Tylerian Evaluation Approach, an objectives-oriented 
evaluation model. This approach was selected since the broad goal of the program was 
identified and the focus of the evaluation was to determine if and to what extent this goal had 
been met (Worthen, Sanders, & Fitzpatrick, 1997). This firamework identifies discrepancies 
between performance and objectives. The modified model used the following steps; (1) 
articulation of the broad goal, (2) restatement of program components as behavioral 
obj ectives, (3) identification of situations where achievement of objectives can be 
demonstrated, (4) selection of measurement instruments, (5) data collection, and (6) 
comparison of data results with objectives. 

Validity and Reliability 

Survey instruments were selected which had demonstrated high levels of validity in 
their development and factor analyses, and high levels of internal consistency reliabilities for 
total scale and sub-scale analyses. Test scores utilized by the schools were obtained from 
instruments that yielded high levels of validity and reliability. 

To increase validity, the evaluators used multiple sources of data. Surveys were used 
to obtain information, workshop evaluations gave insight into what participants found useful 
and effective, and interviews provided depth and insight into the context in which the 
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program was conducted. Triangulation was obtained through analysis of these multiple 
sources of data (Stake, 1995). 

Ensuring that data collection methods were consistent throughout the project and 
between the evaluators increased reliability. Additionally, reliability of the qualitative portion 
of this evaluation was addressed using standard procedures in the field such as defining the 
reporting of the methods, utilization of a theoretical fi-amework to guide the study, and 
utilization of multiple evaluators (Seale, 1999) 

^ , Data Sources and Data Collection 

The School Improvement Progress Needs Assessment Questionnaires. A review of 
the literature was conducted and theses questionnaires were identified as best meeting the 
needs of the evaluators. The questionnaires were developed by the Norfolk (VA) Public 
Schools to facilitate a School Improvement Program (Dodson, 1984). These questionnaires 
were designed for six respondent groups (all stakeholders in the school improvement 
program): (1) parents, 42 items, (2) pupils, 50 items, (3) teachers, 85 items, (4) 
administrators, 75 items, (5) special program/resource teachers, 75 items, and (6) a\^iliary 
personnel, 40 items. Each questionnaire was composed of six sub-scales: (1) admuustrative 
leadership, (2) high expectations, (3) school climate, (4) basic skills, (5) monitoring pupil 
progress, and (6) resources. The coefficient alpha reliability estimates ranged fi’om .86 to .97 
for the six questionnaires, averaging .95. 

Sets of student questionnaires were delivered to each of the five charter schools by 
the evaluators. Student questionnaires were administered by the teachers in the classrooms 
and collected by the evaluators in the principal’s office. Address labels for the parent 
questionnaires were supplied to the evaluators. The evaluators mailed the questionnaire, a 
cover letter fi-om the respective principal, and a stamped, return envelope to parents at four of 
the schools. The return envelopes were addressed to a neutral site. At one of the schools, 
questionnaires were distributed at a parents’ meeting, together with the stamped return 
envelope. Questionnaires for the teachers, principals, special programs/resource teachers, and 
auxiliary personnel were distributed at the schools together with the stamped return envelope 
for their responses. Surveys were coded to identify the respective charter school represented, 
but not to identify the respondent. 

Workshop Evaluations. Throughout the year, five staff development workshops were 
conducted for program participants. A three-day workshop started the year in August, 
followed by a one-day workshop in November, and three two-day workshops in January, 
March, and April. Program personnel developed an evaluative instrument to query workshop 
participants on various aspects of the program, including the quality of the presentation, 
content, handouts, and usefulness for the classroom. In addition, comments were elicited 
fi’om the participants regarding any phases of the workshops on which they wished to express 
their views. 

The project coordinator was responsible for administering the workshop evaluations 
at the close of each of the five workshops. The completed evaluation forms were mailed to 
the evaluators for analysis. 
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Project Correspondences. Throughout the year, the evaluation team was kept abreast 
of project activities through regular correspondences from the project coordinator and 
throu^ numerous discussions regarding the project. 

On-site Visits and Steering Committee Interviews. The background or context for the 
evaluation was developed from on-site visits to the five charter school campuses participating 
in the project. For the on-site visits and interviews, two evaluators visited each of the five 
charter schools. Prior to the visits, the evaluators met to develop a series of objectives for the 
visit, including a set of common questions that were posed at each of the five sites. At each 
site a member of the steering committee, generally the principal, was interviewed about the 
project and its implementation at that school. These on-site visits provided an opportumty to 
gain additional insight and enhanced the validity and reliability of the subsequent descriptive 
analyses. 

FINDINGS 

The following section presents the findings of the evaluation for the first year of the 
Aimenberg funded grant, “Comprehensive Accountability, Professional Development, and 
Assessment System for the Tri-County Charter School Partnership.” These findings are built 
on the data collected and relevant literature reviewed. Prior to presenting the findings, the 
context in which the project was initiated is presented. Context as described here refers to the 
environment, which is also sometimes termed climate or culture of the school (Silverman, 
1993). As mentioned earlier, context and organizational support are critical elements in 
introducing any change into a school. Therefore, understanding context is essential to 
assessing the success of the project. Following a discussion of the context that framed this 
project, the findings are presented in response to the evaluative questions and sub-questions. 



Context 

The evaluators, to better understand the context in which the project was being 
implemented, conducted interviews during the on-site visits. Using a semi-structured 
interview protocol, a team of two evaluators met with a member of the Steering Committee at 
each school. The first interview question focused on obtaining the participant’s perceptions 
on charter school effectiveness and the effectiveness of their charter school. The second 
question focused on perceptions of professional development (a key element in project 
implementation) and the participant’s perception of the current professional development 
activities. The questions were: 

1. a. In your opinion, what represents an effective charter school? 
b. In what ways is your school effective? 

2. a. In your opinion, what makes a professional development program effective? 
b. How would you describe the effectiveness of the current professional 
development program? 

In response to the first part of question 1, “In your opinion, what represents an 
effective charter school?” several common themes emerged. Interviewees from three schools 
stressed the importance of support and commumcation with parents. For example, one 
administrator stated that an effective charter school has good “parental buy-in. Another 
principal who commented that at such a school, “parents are very involved” reinforced this. 
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Another concept mentioned by interviewees at three schools focused on the 
importance of meeting students’ individual needs. They described an effective charter school 
as one that motivates children to achieve potential by meeting the needs of individual 
children. It was also stated that these schools have a multi-age/multi-level program, which is 
tailored to the individual child. 

A third area of agreement for three of the school admimstrators was the importance of 
community involvement. They believed that effective charter schools are very commumty- 
oriented. The mission should be to provide an "enriched, iimovative educational program and 
a positive safe environment for students." 

Three interviewees stated that good charter schools have good staff members and 
good teachers. For example, one principal noted the importance of “quality teachers while 
another stated that an effective charter school “puts teachers on a pedestal, and teachers are 
very receptive to the school’s mission. 

Two school principals pointed out that competent budget and financial management 
is important. As an example, one principal said that an effective charter school must be well 
managed with financial systems in place. 

Two school interviewees felt that high academic standards are important. As one 
principal pointed out an effective charter school will “demonstrate high academic standards 
and expectations.” 

Several concepts for effectiveness were mentioned by one of the interviewees. These 
included cooperation with the board of education, professional development, safety, 
character development, integrity, no discrimination, vision and mission, and family 
atmosphere. 

In responding to the second part of the question, “In what ways is your school 
effective?” all five of the interviewees agreed on two common themes. The first was that 
their schools have academic integrity. One principal described her school as a school that 
practices “high standards of academic integrity,” and another administrator said that the her 
school has a focus on “core knowledge.” Still another principal described her school as a 
place with a focus on learning styles. Finally, another principal reported that 100% of the 
students in his school passed the Florida Writes test. 

The second area of agreement by all five interviewees was the importance of the 
involvement of parents and the commumty. Admmistrators reported inviting parents into the 
school through volunteering. As one principal stated “parents share time, talent, and money. 
Some taught for FCAT prep. Parents empowered to be part of SIP input on strategies.” The 
relationship between the schools and the parents were summed up nicely by a principal who 
said, “. . .part of the mission is to involve parents. Our parents are our community.” 

Two school interviewees commented about the importance of having a sound 
organization, having an evaluation and planning system, and teaching character education. 
Single interviewees discussed several concepts regarding their schools. These included vision 
and mission, no discrimination, safety, technology, and commumcation with stakeholders. 

The second question posed to the interviewees focused on professional development 
programs in general. “In your opinion, what makes a professional development program 
effective?” In response to this question, no common themes emerged. One interviewee did 
not respond to this question. Although the other four interviewees did not mention common 
concepts, they did offer ideas. Concepts mentioned by one interviewee included teacher buy- 
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in, learning communities, communication with staff, understanding of school’s vision and 
mission, awareness of cultural differences, teacher empowerment, and assessment. 

In response to the final question, “How would you describe the effectiveness of the 
ciarrent professional program?” several common themes emerged. Interviewees from three 
schools commented that resources were adequate. However, one administrator felt that the 
challenge was getting all counties to work together. For example, he noted that planning days 
were different across counties. He also commented that a positive element is that schools get 
to communicate with other schools. 

Another area where three interviewees concurred was that teachers have written 
performance assessments. One principal reported that “teachers have already written 
performance assessments and are ready to implement. Initially, they thought of the 
experience as ‘tedious’; however, now they see the merit.” 

Three interviewees discussed attendance. One administrator reported asking for "huy 
in" from teachers because attendance was not "compulsory." Teachers may not be required to 
participate because of the union contract. He added that teachers from the areas of physical 
education, social studies, and foreign language dropped out although a bonus of $1,200 was . 
tied to the professional development. Initially, attendance was not compulsory, but one of the 
principals stated that she now required teachers to attend professional development 
workshops. Two interviewees agreed that the time frame for the professional development 
has been drawn out. Some teachers got bored because each charter school is different. 
Teachers have voiced they want to go to the next level. 

The data from these interviews indicates that effective charter schools were viewed as 
those having strong parental and community support, a caring and capable faculty, and high 
standards. Additionally, teaching and learning in these schools is geared to the individual 
learner. All of the charter schools that participated in this project expressed their commitment 
to academic integrity and high standards. They also unanimously articulated the importance 
of the support they receive from parents and the commumty. 

While it is interesting to note that no common description of effective professional 
development programs was found, all of the schools felt that the current project provided 
elements that have been previously described (U.S. Department of Education, 1994) as key to 
effective professional development efforts. These include an extended commitment of time 
and resources, development of a framework to teaching to high standards (written 
performance assessments), collaboration between teachers and facilitators, and teaching 
effectiveness and student learning as the basis for measuring success. 

Evaluation Questions and Sub-Questions 

In the following sections the findings of the evaluation are presented in context of each of the 
6 Research Questions. 

Evaluative Question 1. Did the program provide the opportunities for professional 
development activities that prepare teachers and administrators with the skills necessary to 
develop a standards-based accountability system and performance assessments aligned to 
these standards? 

The program provided a yearlong series of workshops and professional development 
opportunities for teachers and administrators within the participant schools. Staff from the 
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“Center for Performance Assessment” facilitated these activities and provided on-site 
opportunities for faculty professional development. These school visits provided teachers and 
administrators with the opportunity to interact on an individual school basis with staff 
members who could provide assistance with project activities. 

As mentioned previously, schools were free to select their own level of purticipation 
in these activities. While some schools chose to make these professional development 
activities a mandatory requirement for their faculty, other schools sought participants on a 
voluntary basis. 

Sub-question 1.1. Did teachers and administrators attend the professional 
development workshops? Overall, charter school participation in the workshops was 34%. 
The most popular workshop attended (60%) was on “Data Driven Decision Making” in 
November. The lowest participation rate (20%) was noted on the first day of the “Advanced 
Seminar in Standards Based Performance Assessment” in April. Overall teacher attendance 
was 36%; overall principal attendance was 21%. These workshops were held throughout the 
year while man y other school-related obligations were being met and teacher participation at 
some schools was voluntary, this probably would account for these attendance rates. 

Sub-question 1.2. Did participants perceive that the professional development 
workshops provided them with the information they needed to meet program objectives? 
Tallies from the workshop evaluations indicated that the participants felt (1) the material was 
presented in a clear, concise, and easy to understand manner, (2) the material presented was 
relevant to their classroom teaching, (3) new information and insights were presented, (4) the 
handouts would be useful to their teaching, (5) much of what they had learned will be applied 
in their classrooms, and (6) they now have a much better idea of what their roles will be in 
implementing the Annenberg grant at their schools. In addition, (7) they feel comfortable 
about training their colleagues in sharing what they learned at the workshops. 

Sub-question 1.3. What other forms of professional development activities did the 
participants feel would be helpful in meeting program objectives? Comments on the 
evaluation forms for the five professional development workshops indicated that the 
participants generally appreciated what was presented in the workshops and how it was 
presented. The most prevalent theme in their comments was more “hands on” workshop time 
to help participants in developing the kinds of lessons and performance assessments they’ll 
need to facilitate the performance assessment systems being developed. 

Evaluative Question 2. Did each charter school align their curriculum with the 
Florida Sunshine Standards and other agreed upon standards and benchmarks? 

Each of the participant schools aligned their curriculum to the Florida Sunshine 
Standards. Benchmarks that quantify standards were also completed by the 4 elementary 
schools and were almost completed at the middle school level. 

Sub-question 2.1. Were power standards identified? During the third quarter, 60% of 
the schools had identified and prepared a first draft of power standards in all content areas 
and grade levels. In addition, these schools were able to share this information with their 
teaching staff. During the fourth quarter, the principals of the elementary charter schools met 
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and identified a common list of power standards fi:om the Florida Sunshine State Standards. 
The middle school had prepared the first draft of their standards. 

Evaluative Question 3. Did each charter school create and utilize appropriate 

classroom performance assessments? 

Most schools had created and utilized appropriate classroom performance 

assessments. 

Sub-question 3.1. Were customized performance assessments purchased from the 
Center for Performance Assessment? 

The Center for Performance Assessment no longer provides customized performance 
assessments as a service to its clients. This objective was modified and the Center helped 
schools develop their own customized performance assessments. 

Sub-question 3.2. Were the IP AS purchased? Schools received 192 model student 
performance assessments from the Center for Performance Assessment and were encouraged 
to select and adapt these assessments for use in their classrooms. 

Sub-question 3.3. Were the IPAS utilized by each charter school? Four (80%) of the 
schools reported using one of the EPAS at different grade levels. 

Sub-question 3.4. Did each charter school create their own customized 
performance assessments? At the end of the academic year, 100% of the schools developed 
a custom made performance assessment with power standards. 

Sub-question 3.5. Did each charter school utilize their customized performance 
assessments? Three schools (60%) reported using the first custom made performance 
assessment in the classroom at the grade level for which it was designed. 

Evaluative Question 4. Was a Standards Based Progress Report created and 
implem ented? 

The standards based progress report is still in the developmental stage. Recent 
discussions have begun to address this matter. The creation and implementation of the 
standards based progress report will be a critical need to be addressed and resolved in year 

two. 

Evaluative Question 5. Was a computer based student progress and standards 
reporting system purchased and utilized? 

To date, no student progress and standards reporting system has been either identified 
or developed. This is a critical need that is to be addressed and resolved in year two. 

Evaluative Question 6. Were stakeholder satisfaction surveys administered to 
students, parents, teachers, administrators, special program/resource teachers, and 
auxiliary personnel? 

The School Improvement Progress Needs Assessment Questionnaires, previously 
discussed in this report, were administered to the six groups listed above (all stakeholders in 
the school improvement program). The response rate for each of the respondent groups was: 
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(1) students, 79%, (2) parents, 31%, (3) teachers, 38%, (4) administrators, 89%, (5) special 
program/resource teachers, 38%, and (6) auxiliary personnel, 35%. 

Sub-question 6.1. What were the perceptions of the school of students, parents, 
teachers, administrators, special program/resource teachers, and auxiliary personnel? 

The following section presents a brief summary of the findings fi-om these surveys. 
The student questionnaire has a simple "Yes or No" response scale. The remaining five 
questionnaires all use a 5-point Likert response scale (l=Strongly Disagree, 2— Disagree, 
3=Undecided, 4=Agree, 5=Strongly Agree). Along with the frequency (N) and percentage 
responses (%), the average score (mean) and standard deviation (SD) is reported for each 
item. For items that are negatively stated, the optimal response is "Strongly Disagree." The 
scoring for these items is the reverse of the 5-point scale above. 

Student Questionnaire Results. A total of 793 usable questionnaires were returned 
for students at the five charter schools. In review of the responses to the items, students were 
quite positive in their perceptions of the school. The highest rated items, those receiving 95% 
or more responding favorably were: "my teacher expects me to complete my work," "my 
teacher knows that I can do things right," and "the principal wants me to be successful. 

Other than the "library item," the lowest rated item was "my teacher does not grade 
homework assignments." 

Parent Questionnaire Results. There were 335 parent questionnaires mailed back 
from the five charter schools, hi general, parents were quite positive in their perceptions of 
the charter school their children were attending. For parents, the highest rated item regarded 
the statement, “this school is a safe place in which to learn and work.” These charter schools 
evidently fulfill parents’ wishes for safe keeping for their children. The lowest rated item was 
“the building and grounds are adequate for the school’s program.” This rating reflects the 
conditions under which these charter schools have opened in quite meager facilities in 
contrast to other schools parents. 

Teacher Questionnaire Results. Twenty-one teacher questionnaires were returned 
from the five charter schools, representing a 38% response rate. The item means for teachers 
were generally quite positive, but not at the level observed for parents. The highest rated 
items indicated “teachers at this school treat children as persons” and “students in the school 
were being trained in test-taking skills.” The lowest rated teacher response was to the 
statement “at this school the number of low-income children retained in a grade is 
proportionate to the number of other children retained in that grade.” 

Administrator Questionnaire Results. From the responses to the administrator 
questionnaire, these charter school leaders are positive about their programs, personnel, and 
students. This questionnaire had the highest response rate 89%. On 8 of the 75 items, 
administrators were unanimous in their optimal ratings of the characteristic being queried. 
These items included three (3) statements about themselves, “the principal of this school 
provides clear, strong, and centralized instructional leadership,” “the principal of this school 
holds high level of expectations for all teachers,” and “generally, principal-teacher 
relationships at this school are very good,” and three (3) statements about their teachers. 




12 



11 



“teachers in this school pay as much attention to slow students, hy smiling and making eye 
contact as often with them,” “when students are assigned seatwork, teachers in this school 
monitor their efforts closely,” and “teachers in this school make a conscious effort to involve 
every student in class discussions and exercises,” and two (2) statements about school policy, 
“every teacher in this school has access to written school-wide goals and objectives and a 
specific time schedule has been established for teaching reading and mathematics by each 
classroom teacher.” The single item rated markedly lower than all other items by the 
adrniiiistrators was consistent with the low ratings on the item given by teachers, at this 
school the number of low-income children retained in a grade is proportionate to the number 
of other children retained in that grade.” 

Special Program/Resource Teacher Questionnaire Results. There were six special 
program/resource teacher questionnaires returned for analysis. For the item responses, this 
group appears generally positive about the various aspects at their charter schools. On 6 of 
the 75 items these teachers were unanimous in their optimum appraisal; 5 of these items were 
complimentary of the principal, “the principal is very active in securing resources to facilitate 
instruction,” “the principal of this school provides clear, strong, and centralized instructional 
leadership,” “the principal is highly visible throughout this school,” “the principal of this 
school holds high expectations for all teachers,” and “the principal at this school is accessible 
and responsive to parents” with the sixth item regarding the fauness of handling discipline 
incidents, “students in this school are not punished for observed offenses differently for one 
student than others.” The three lowest rated items were “teachers in this school believe that a 
student’s home background is the primary factor that determines individual student 
achievement,” “a policy that no one disturbs a teacher during instruction on the basic skills is 
enforced throughout this school,” and again, the item, “at this school the number of low- 
income children retained in a grade is proportionate to the number of other children retained 
in that grade.” 

Auxiliary/Professional Personnel Questionnaire Results. There were 22 
questionnaires returned by auxiliary/professional personnel. Overall, this group rated school 
characteristics as generally good, but lower than the other groups using this 5-point scale. 
Five items were rated above 4.5, “this school informs parents regularly of their children s 
academic progress,” “students in this school are expected to master the basic skills,” “the 
principal is an important resource person in this building,” “I am an important member of this 
school’s s taff ,” and “the principal of this school is very accessible to all personnel.” The 
three lowest rated items reflect a variety of concerns, “students in this school abide by the 
school rules,” “most parents would rate this school superior,” and “there are sufficient 
supplies and materials in this school for all personnel to perform their assigned duties.” 

Sub-question 6.2. What are the stakeholder perceptions when the questionnaire 
results are broken down by sub-scale: administrative leadership, high expectations, school 
climate, basic skills, monitoring pupil progress, and resources? 

The School Improvement Progress Needs Assessment Questionnaires can be broken 
down into sub-scale analysis for each of the respondent groups. As shown in the tables that 
follow, each questionnaire item can be classified into one of the six sub-scales. 
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Administrative Leadership. Seventy percent of the student responses were positive 
regarding the principal. In the overall responses for the other respondent groups, ratings were 
quite favorable to adrninistrative leadership with, not unexpectedly, the administrators’ 
ratings being the highest. 

High Expectations. The highest overall sub-scale rating for students was high 
expectations with 86% of the responses being positive. In particular for teachers and 
auxiliary personnel, the overall ratings for high expectations were lower. These low ratings 
are affected by responses to the items “at this school the number of low-income children 
retained in a grade is proportionate to the number of other children retained in that grade” 
and “teachers in this school believe that a student’s home background is the primary factor 
that determines individual student achievement.” 

School Climate. For students, 74% of the ratings on school climate were positive. For 
the other respondent groups, school climate was rated generally good. For students, the 
highest rated item (90%) on school climate was “I like my teacher.” For the other respondent 
groups, parents liked the school’s safety, understood the policies, and reported teachers make 
themselves accessible to parents. Teachers felt they treated children as persons and said 
parents felt welcome when they visited the school. The administrators optimally rated 
principal-teacher relationships. Resource teachers reported students were disciplined fairly 
and auxiliary persoimel rated the principal high on school climate related items. 

Basic Skills. Student ratings of basic skills items were 82% positive. Basic skills were 
generally rated quite positive by the other respondent groups with administrator ratings being 
higher than teachers, auxiliary personnel, and parents. Ninety-two percent of the students 
said, “my teacher gives us tips on learning how to learn” and 89% reported “my teacher 
encourages me to answer difficult questions.” For teachers, averages of the basic skills items 
are centered and about equally split around 4.00. The lowest rated items by teachers on basic 
skills involve school policy, “teachers at this school set aside at least 120 minutes per day for 
basic and content teacher-directed reading” and “a policy that no one disturbs a teacher 
during instruction of the basic skills is enforced throughout this school.” 

Monitoring Pupil Progress. Student responses to the monitoring pupil progress sub- 
scale items were 73% positive. For the other respondent groups, overall responses on 
monitoring pupil progress were quite positive with teachers having the lowest average. The 
student responses to these sub-scale items are quite positive about being taught to take tests 
(92%), having the teacher talk to them about lessons and grades (88%), and in the teacher 
t alkin g to their parents about their schoolwork (86%). However, it should be noted that 51% 
of the students reported their teacher does not grade homework assignments and 42% 
reported their teacher never tells them how they do on tests. Administrators reported 
unanimously “when students are assigned seatwork, teachers in this school monitor their 
efforts closely.” The lowest rated items by teachers were four items about using standardized 
test results for evaluating student progress and for evaluating the effectiveness of the 
instructional program. 
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Resources. The lowest overall rating by students on the six sub-scales was for 
resources with only 62% of the student responses being favorable. For the other respondent 
groups, teachers and parents rated resources lower than did the administrators, resource 
teachers, and auxiliary personnel. Student responses to the resources sub-scale items may 
reflect some shortage of books and supplies. Some teachers and auxiliary personnel report 
there is a lack of sufficient quantities of supplies and materials for them to perform their 
assigned duties. The administrator responses to the resource sub-scale items were generally 
quite favorable. 

FINAL COMMENTS 

In summary, the partnership was successful in establishing the groundwork for year 
two activities. Specifics strengths and limitations of the project after year one have been 
identified through the comprehensive evaluation. 

Throughout year one of the project, the charter school partnership collaborated in 
providing various professional development activities to acquaint the participants with the 
requisite skills needed to complete the project. Attendance in the training sessions was 
limited due to the voluntary nature of the project at some schools and the many other school- 
related obligations participants were involved in. Overall, the training sessions were well 
received and provided insight into the project objectives and development of performance- 
based assessments. Thus the teachers and administrators who did attend all or a majority of 
the workshops had the advantage of being able to integrate these new skills into their 
teaching repertoire (Arrends, 1998; Joyce & Showers, 1995). 

The charter schools aligned their curricula with the Florida Sunshine Standards. The 
schools identified power standards and are developing related performance assessments. 

Each school has piloted performance assessments at various grade levels. This commitment 
to building a framework to assess student performance is a key indicator of an effective 
accountability system (U.S. Department of Education, 2000) and an important component of 
this project. 

While no computer-based student progress and standards reporting system has been 
identified or developed, it is recognized that a student database is essential for tracking 
student performance on standardized tests and performance assessments. 

Stakeholder surveys of students, parents, teachers, administrators, and auxiliary 
personnel generally were very positive about the schools, programs, teachers, administrators, 
and relationships between the various stakeholder groups. This is extremely important since 
the literature indicates that context and relationships are important elements in implementing 
successful programs (Chemiss, 1988; Schwahn & Spady, 1998). In addition, much insight 
into the climate of these charter schools was gained from the questionnaire tallies and 
breakdowns into six sub-scales: administrative leadership, high expectations, school climate, 
basic skills, monitoring pupil progress, and resources. Since the principal’s support of new 
programs is key to success (Fullan, 1992; Yukl, 1994), feedback from these surveys provided 
a rich source of data to reinforce the need for change. 

An important component of this accountability system is student achievement (U.S. 
Department of Education, 2000). Currently, no comprehensive accoimtability system exists 
within the charter school network. This project has provided the framework for the 
partnership schools to focus on developing and implementing a replicable, comprehensive 
accountability system in each of the schools during year two. 
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